Training evaluation: how to measure the effectiveness of eLearning
One of the main issues with training and eLearning is related to the evaluation of the effectiveness of the learning process: Kirkpatrick Four Levels model and its evolution, the Phillips ROI Methodology, as well as the Brinkerhoff’s Success Case Method can help with. Learn all about these models and how to use them in the article.
THE KIRKPATRICK FOUR LEVELS
Donald Kirkpatrick developed his four levels model to measure training effectiveness for his PhD dissertation in the mid-Fifties and today it is the most widely used model for this purpose.
The model is based on many variables that take into account four different aspects of training. Level one is the participants’ reaction to the training, namely what perception students have about their training experience, collected by surveys to be completed at the end of a course. This level of evaluation is obviously quite subjective, but can still help to collect useful insight to improve training delivery and spot possible issues, especially in eLearning.
Level two of the model regards what participants have learnt in the course. This can be measured by post-tests or simulations and assignments, designed to make sure that all the students have reached the set learning objectives in terms of knowledge, skills and attitudes.
The following level, the third, takes into account the new behaviours developed as a result of the training. In particular, it measures whether students apply what they have learnt on the job, by surveys or direct observation.
The fourth and last level evaluates which business results have been achieved, in terms of actual improvements, thanks to the training.
It is important to underline that the Kirkpatrick model works best when training outcomes to measure are relatively isolated from other factors, since this allows to have a higher degree of accuracy. On the other hand, to be able to isolate the impact of training it may be necessary to do extra work, for example pre-testing participants to confront results with post-tests, which may not be always worthwhile to do. To use this model in an effective way, then, it can be useful to work backward from level four to one, relating desired business goals to the training.
THE PHILLIPS ROI METHODOLOGY
Jack Phillips improved Kirkpatrick’s four-level model adding a financial component, in particular with two more levels. Level zero represents program inputs, like the number of students enrolled in the course or the cost of the training. Connected to this, level five is eventually the ROI – Return On Investment – which helps to measure the financial impact of the training, comparing it to analyzed inputs and therefore isolating it from other factors.
BRINKERHOFF’S SUCCESS CASE METHOD
A different point of view is represented by Brinkerhoff’s Success Case Method, which addresses the issue of results variation within the class. This method aims to identify what makes some people more successful than others, in order to spread that behaviour to everyone, using an impact model: starting with the business goals for the training and moving backwards, to clearly highlight how training can benefit the business.
Compared to other models, Brinkerhoff’s Success Case Method helps with two main recurrent issues: students don’t always implement everything they have learnt in training and, on the other side, many external factors can negatively influence training results. This method can help to make the most of training programs, improving its value for participants and businesses.
OTHER EVALUATION TECHNIQUES
In addition to the models we have described, sometimes it may be necessary to only focus on one specific piece of information in relation to the training.
For example, the evaluation could be based on performance, looking at skills participants can demonstrate on the job after the training.
On the other hand, it could be useful to apply for external recognition, like industry best practice awards, which can help to have a more objective evaluation of results and also promote training sponsorship within the company.
Another trend in training evaluation regards the application of predictive analytics: instead of looking backward, this approach aims to influence the results of the program before they happen, by using collected data to improve future training.
Of course, there are many different ways to evaluate training, according to the desired goals and context: it is important to choose the most appropriate one that helps you generate useful insight, to keep training under control and make sure that it is effective.
Do you want to know more about training? Read our next articles!